Skip to content

Fix github tests suites#5

Merged
sebastiw merged 17 commits into
developfrom
fix/ci-resurrection
Jun 24, 2026
Merged

Fix github tests suites#5
sebastiw merged 17 commits into
developfrom
fix/ci-resurrection

Conversation

@sebastiw

@sebastiw sebastiw commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Restores the GitHub Actions Test and Examples workflows for rules_erlang on top of Bazel
8.4.2 (the version shipping with current bazelisk) and Bzlmod, fixes the rule regressions
exposed along the way, and triages the remaining peripheral CI jobs.

Both workflows are green on e753ad8.

Bazel 8 / Bzlmod compatibility

  • 8faae61 Pin .bazelversion to 8.4.2. Add allow_empty = True to glob(...) calls
    that became errors under --incompatible_disallow_empty_glob. Migrate sh_test usage to
    @rules_shell//shell:sh_test.bzl (removed from core). Update CT/eunit fixtures and
    rebar_config_to_erlc_opts to match the testing-support contract from 228b062.
  • ca9bd9c Fix dialyze and root-level toolchain resolution under bzlmod:
    • private/ct.bzl::code_paths: stop prepending dep.label.workspace_root to short paths that
      already start with ../<canonical_repo_name>/. Under Bzlmod the runfiles tree has no
      external/ directory, so the previous logic produced
      external/rules_erlang+/../rules_erlang+/... and dialyzer failed with "No such file,
      directory or application". Fixes //erl_attrs_to_json:dialyze,
      //rebar_config_to_erlc_opts:dialyze, and //rules_erlang_compiler:dialyze.
    • tools/rules_erlang_compiler/src/types.hrl: add the test_modules key to target() and
      target_extended(). Commit 228b062 made dot_app_file:render/3 destructure this key but
      never updated the typespec.
    • MODULE.bazel + repositories/erlang_config.bzl: generate an erlang_external_platform
      (parented on @platforms//host) in @erlang_config and register it via
      register_execution_platforms in the root module. After bfb48d7 made erlang_internal
      the default constraint value, the host platform no longer satisfied the external toolchain's
      exec_compatible_with = ["//:erlang_external"] requirement.

OTP default bump

  • 80867f5 Bump DEFAULT_ERLANG_VERSION from 26.0 to 28.1 (with verified sha256). OTP
    26.0 source builds fail on the current macOS 15 runner image.
  • 1ec390f Update OTP-version-dependent test references after the bump.

extract_app rule fix

  • e70ed5a Namespace .beam and .app outputs by ctx.label.name. Previously, having both
    :erlang_app and :test_erlang_app in the same package (as in the umbrella example) caused
    ERROR: file 'apps/greeter/ebin/greeter.beam' is generated by these conflicting actions
    because both targets declared the same ebin/ paths. Header and priv/ files are
    intentionally kept at their original locations to avoid breaking consumers like relx and
    hex-package rules that strip the package prefix when locating resources. Provider data
    (ErlangAppInfo) is unchanged, so all downstream rules (eunit, ct, dialyze, xref,
    escript_archive, runtime symlink tree) continue to work via the provider.

CI workflow restoration & triage

Test workflow

  • 5553478 Drop legacy --noenable_bzlmod jobs (unit-test, test) and remove from
    summary.needs. Their WORKSPACE files were deleted in dc6c4b7; Bazel 8 disables
    --enable_workspace by default and WORKSPACE is being retired in Bazel 9. Coverage is
    preserved by the existing unit-test-bzlmod and test-bzlmod jobs.
  • f9998f2 Switch unit-test-bzlmod to bazelisk build //... (the root module has no test
    targets) and silence the platforms version warning.
  • d98aa61 Drop the Windows job entirely (Windows support was already removed in
    dc6c4b7); wire BUILDBUDDY_API_KEY into macOS and coverage jobs so they share the cache.
  • 1d8a98a + fcc21bd Add a check-buildbuddy-secret guard job that probes the
    secret in a step (secrets.* cannot be used in jobs.<id>.if) and exposes has-key as a
    job output. test-bzlmod-internal-erlang gates on it so forks/branches without the secret
    skip cleanly instead of failing the workflow.
  • e753ad8 Add if: ${{ !cancelled() }} to the summary job so it still runs (and gates
    on the jq check, which already accepts skipped) when test-bzlmod-internal-erlang is
    skipped. Previously GitHub auto-skipped summary through needs, defeating its gating role.
  • Also dropped test-host-erlang-change-detected (the :otp_version target and erl_eval.bzl
    rule it depended on were removed in dc6c4b7) and pinned test-bzlmod-macos to
    runs-on: macos-15 (was macos-latest, now Tahoe).

Examples workflow

  • fcc21bd Bump runs-on from ubuntu-20.04 (retired by GitHub in April 2025; jobs sat in
    the queue indefinitely) to ubuntu-24.04.
  • 4696228 Repoint at examples/umbrella (the only remaining example after dc6c4b7
    deleted examples/basic). Renamed the job from examples-basic to examples-umbrella and
    updated the artifact name accordingly.

Umbrella example fixes

These were latent bugs that surfaced once the Examples workflow could actually run:

  • 5d3d945 Use ct_suite instead of ct_test for umbrella_smoke_SUITE in
    apps/greeter_web/BUILD.bazel. ct_test was being handed a raw .erl suite that wasn't
    being compiled; ct_suite invokes the proper erlang_bytecode step so the suite is
    available in the expected test/ directory.
  • fc1d7ef Add eunit and common_test to the dialyze PLT apps in both apps/greeter
    and apps/greeter_web. DEFAULT_PLT_APPS only covers erts/kernel/stdlib; the
    *_tests.erl modules call eunit:test/1, so dialyzer reported Unknown functions: eunit:test/1 and exited with warnings.

Other

  • f6f3b4f, bf96348 Initial test-setup tweaks and upload-artifact@v7 bump.
  • 3a950b3 setup-beam ImageOS override on windows-2025 (later mooted by d98aa61
    dropping the Windows job).

Validation

  • bazelisk test //... (root, bzlmod): builds successfully against the external host toolchain
    via the newly registered erlang_external_platform.
  • cd test && bazelisk test //...: 17/17 tests pass locally.
  • cd examples/umbrella && bazelisk test //...: all targets including dialyze pass locally.
  • CI on e753ad8: both Test and Examples workflows pass; test-bzlmod-internal-erlang
    correctly skipped on the absence of BUILDBUDDY_API_KEY, summary runs and validates.

Notes for review

  • The DEFAULT_ERLANG_VERSION bump is a behaviour change for anyone using
    internal_erlang_from_github_release() without specifying a version. Pinning to an older
    OTP can still be done per-module by passing version = "...".
  • The extract_app namespacing is a path change in bazel-bin/ only; the ErlangAppInfo
    provider surface is unchanged, so consumers that go through the provider (which is all
    in-tree rules) are unaffected.
  • macos-15 was picked over macos-26 as the conservative choice.

sebastiw added 3 commits June 23, 2026 14:59
… regressions

The repository's GitHub Actions workflow had been failing for some time due
to a combination of Bazel version drift (last_rc -> 9.1.1rc1), breaking
changes in Bazel's glob and sh_test defaults, and regressions in the rule
implementations and test data.

Infrastructure (Bazel 8.4.2 + stricter glob/sh_test defaults):
- .bazelversion: last_rc -> 8.4.2
- erlang_app.bzl: allow_empty=True on 9 native.glob sites (include/, etc.)
- ct.bzl: allow_empty=True on the 2 globs in ct_suite/ct_suite_variant
- test/MODULE.bazel: platforms 0.0.7 -> 0.0.11; add rules_shell 0.6.1
- test/app_file/BUILD.bazel: switch app_file to public load; load sh_test
  from @rules_shell (no longer in @bazel_tools by default)

CT/eunit regressions introduced in 228b062 "Fix testing support (ct, eunit)":
- private/ct.bzl: flat_deps(deps + compiled_suites) -> flat_deps(deps);
  compiled_suites are .beam files, not ErlangAppInfo providers, so passing
  them through flat_deps causes an analysis-time error
- private/ct.bzl: ct_run -dir ebin -> -dir test; the ct_suite macro still
  compiles SUITEs to dest="test", so -dir ebin found no suites

Test data updates for product changes:
- test/shard_suite/BUILD.bazel: erlang_bytecode2 -> erlang_bytecode v1
  (v1 accepts string-list erlc_opts); replace eunit(compiled_suites=...)
  (no such attr) with test_beam=[...] on test_erlang_app + eunit_mods
- test/rules_erlang_compiler/test/dot_app_file_SUITE.erl: add
  test_modules => [] to the Target map (228b062 made the key mandatory
  in dot_app_file:render/3 but did not update the test)

Product bug fix:
- tools/rebar_config_to_erlc_opts/src/rebar_config_to_erlc_opts.erl:
  honor no_debug_info in rebar.config erl_opts by dropping the default
  +debug_info from the output when no_debug_info is present; also
  lists:flatten the io_lib:format result so output entries are flat
  strings rather than nested iolists like [[43],"deterministic"]

Local result with Bazel 8.4.2: 13/13 tests pass in test/.
@sebastiw sebastiw changed the title Fix github actions Resurrect CI: pin Bazel 8.4.2, fix glob/sh_test defaults, fix CT/test regressions Jun 23, 2026
sebastiw added 7 commits June 23, 2026 16:03
The unit-test and test jobs ran bazelisk with --noenable_bzlmod, but the
WORKSPACE files were removed in dc6c4b7 "Remove unused rules, tools, and
examples". With Bazel 8 the previous setup failed at startup with:

  ERROR: Both --enable_bzlmod and --enable_workspace are disabled, but one
  of them must be enabled to fetch external dependencies.

Adding --enable_workspace would not help because there is no WORKSPACE
file to fall back to, and WORKSPACE support is being retired in Bazel 9.

The unit-test-bzlmod and test-bzlmod jobs already cover the same OTP
25.3 and 26.2 matrix via bzlmod, so the legacy jobs are dropped and
removed from the summary needs list.
Three independent fixes that were preventing the bzlmod test jobs from
passing under Bazel 8.4.2:

1. private/ct.bzl: code_paths

   The helper was joining `dep.label.workspace_root` with short
   dirnames that already start with `../<canonical_repo_name>/`. In a
   bzlmod runfiles tree there is no `external/` directory at all, so
   the resulting path `external/rules_erlang+/../rules_erlang+/...`
   does not exist and dialyzer fails with "No such file, directory or
   application". Short paths already contain the correct runfiles
   prefix for files from other repositories, so the workspace_root
   should not be prepended.

   Affects //erl_attrs_to_json:dialyze,
   //rebar_config_to_erlc_opts:dialyze, //rules_erlang_compiler:dialyze.

2. tools/rules_erlang_compiler/src/types.hrl

   Commit 228b062 ("Fix testing support (ct, eunit)") made
   dot_app_file:render/3 destructure `test_modules` out of the target
   map but never updated the `target()` (and `target_extended()`)
   typespec, causing a dialyzer "Invalid type specification" error.

3. MODULE.bazel + repositories/erlang_config.bzl

   Since bfb48d7 ("Revert "Revert "default to internal"") the default
   value of the `erlang_internal_external` constraint setting is
   `erlang_internal`. That works for test/MODULE.bazel which uses
   internal_erlang_from_github_release, but the root MODULE.bazel
   uses external_erlang_from_path (auto-detected via _default_erlang_dict),
   so the host platform no longer satisfies the toolchain's
   `exec_compatible_with = ["//:erlang_external"]` requirement and
   resolution fails with "No matching toolchains found for types:
   //tools:toolchain_type" in unit-test-bzlmod.

   Generate an `erlang_external_platform` (parented on @platforms//host)
   in the @erlang_config repo and register it from the root MODULE.bazel
   via register_execution_platforms. This makes the external toolchain
   resolvable at the root without changing the new "default to internal"
   semantics that the test/ jobs rely on.
- Bump DEFAULT_ERLANG_VERSION from 26.0 to 28.1 (with updated sha256).
  OTP 26.0 source builds fail on the newer macOS 15 runner image, and
  pinning ancient releases for everyone is undesirable. 28.1 is the
  current stable release and builds cleanly on Linux and macOS.

- test-bzlmod-macos: pin runs-on to macos-15 (was macos-latest, which
  has moved to macos-26 / Tahoe). macos-15 / Sequoia is the conservative,
  well-tested image; revisit once macos-26 has had more bake time.

- test-bzlmod-windows: pin runs-on to windows-2025 (was windows-latest,
  whose new image is not yet recognised by erlef/setup-beam), and bump
  the matrix OTP to 28.1 to match the new default.

- Drop test-host-erlang-change-detected entirely. The job depends on the
  :otp_version target and the erl_eval.bzl rule, both of which were
  removed in dc6c4b7 ("Remove unused rules, tools, and examples"). The
  job is also removed from the summary needs list.

test-bzlmod-internal-erlang is left alone: it relies on a BuildBuddy
API key that's available in repository secrets.
…N bump

After bumping DEFAULT_ERLANG_VERSION to 28.1, two test/ artifacts still
referenced the old major version and broke analysis / runtime:

- test/BUILD.bazel + test/.bazelrc: rename the `erlang_26_platform`
  alias (and the rbe --platforms flag pointing at it) to
  `erlang_28_platform`. `@erlang_config//:erlang_<major>_platform`
  targets are only generated for the OTP majors that are actually
  registered, so with the default internal Erlang now at 28.1 the
  alias' `actual` did not exist and `bazelisk test //...` failed
  during analysis for every job.

- test/custom_vars/custom_var_test.sh: the test was checking that the
  OTP_VERSION toolchain variable matched `2[4-6]\.`, which only ever
  served to confirm the variable was populated. Relax the regex to
  `[0-9]+\.[0-9]+` so it accepts any modern OTP major (now exercising
  25.3, 26.2, 28.1 and the host OTP on macOS).

Validated with `cd test && bazelisk test //...`: all 13 tests pass.
The windows-2025 runner image reports ImageOS=win25-vs2026, which
erlef/setup-beam@v1 does not recognise. The action's supported set
includes the bare 'win25', so pass it explicitly via the step's env.
unit-test-bzlmod ran 'bazelisk test //...' at the repo root, but the
root has no test targets (all live in test/, covered by the test-bzlmod
job). Bazel exits non-zero with 'No test targets were found, yet
testing was requested'. The job's purpose is to verify the root
MODULE.bazel resolves and builds, so 'build' is the correct verb.

Also bump the root bazel_dep on platforms from 0.0.10 to 0.0.11 to
match test/MODULE.bazel and silence the --check_direct_dependencies
warning during analysis.
test-bzlmod-windows referenced --@rules_erlang//:ct_test_windows_logdir_drive_letter,
which was deliberately removed (along with all other Windows-specific
code paths) in 413df65 "drop windows". The job cannot be made to pass
without reverting that decision, so remove it from the workflow and
from the summary.needs list.

test-bzlmod-internal-erlang was hitting BuildBuddy's anonymous-access
deprecation (FAILED_PRECONDITION). Wire the BUILDBUDDY_API_KEY secret
into the CONFIGURE BAZEL step and append --remote_header to the
buildbuddy bazelrc config so authenticated traffic is used.
@sebastiw

Copy link
Copy Markdown
Collaborator Author

last one (Test / test-bzlmod-internal-erlang) needs BUILDBUDDY_API key

sebastiw added 4 commits June 23, 2026 18:00
BuildBuddy no longer accepts anonymous traffic, so the job is useless
without the API key. Gate it with a job-level 'if' so forks and any
contexts where the secret is unavailable don't fail CI. Relax the
summary gate to accept 'skipped' alongside 'success' so a skipped
internal-erlang doesn't cascade into a red summary.
The 'secrets' context is not allowed in job-level 'if:' (only github,
needs, vars, inputs are). The previous attempt produced a workflow
syntax error that aborted the entire Test run. Replace the inline
secrets check with a tiny check-buildbuddy-secret job that exports
'has-key' as a job output, then gate test-bzlmod-internal-erlang on
'needs.check-buildbuddy-secret.outputs.has-key == true'.

Examples workflow was stuck queued forever because ubuntu-20.04 was
retired from GitHub-hosted runners. Bump to ubuntu-24.04 to match the
rest of the workflows.
examples/basic was deleted in dc6c4b7 'Remove unused rules, tools, and
examples'. The surviving example is examples/umbrella, which has eunit
and ct_test targets. Update the workflow path, job name, and artifact
name accordingly.
Previously every extract_app target in a package declared its outputs
under <package>/ebin/<file>. That made it impossible to have two
extract_app targets (e.g. a prod :erlang_app and a :test_erlang_app)
in the same package, because both would race to declare the same
ebin/<file>.{beam,app} outputs and Bazel would reject the analysis
with a duplicate-action error.

Namespace only the .beam and .app outputs under <target_name>/ebin/.
The header and priv outputs are left at their natural location: the
package-prefix-stripping helper additional_file_dest_relative_path()
in private/util.bzl expects them there, and shifting them would
double the package segment when those files are later placed into a
runtime erl_libs tree (which manifested as relx failing to find its
priv/templates/* at release-assembly time).
@sebastiw sebastiw force-pushed the fix/ci-resurrection branch from 131a12a to e70ed5a Compare June 24, 2026 02:00
sebastiw added 3 commits June 24, 2026 11:56
The umbrella_smoke_SUITE ct_test invocation never worked since it was
introduced: ct_test() expects the suite beam to already be compiled
into apps/greeter_web/test/, but no rule was producing that beam, so
ct_run -dir test would always fail with 'Suite ... not found'.

Switching to ct_suite() makes the macro emit an erlang_bytecode target
that compiles test/umbrella_smoke_SUITE.erl into the expected test/
subdirectory and threads it through as compiled_suites. ct_suite()
also globs test/<name>_data/** on its own and injects :test_erlang_app
as a dep, so the redundant data= line is dropped.
DEFAULT_PLT_APPS only covers erts/kernel/stdlib. The umbrella's
:test_erlang_app targets pull in test sources (greeter_tests.erl,
greeter_web_tests.erl) that call eunit:test/1, so dialyzer reports
'Unknown functions: eunit:test/1' and exits with warnings. Extend
each app's plt() apps list with eunit and common_test so the
analysis can resolve test-framework calls.
By default GitHub propagates 'skipped' through job dependencies, so the
summary job was being auto-skipped whenever test-bzlmod-internal-erlang
is skipped on forks/branches without BUILDBUDDY_API_KEY. That defeats
the summary's gating role.

Add 'if: ${{ !cancelled() }}' so the summary runs on success / failure
/ skipped (but not on cancellation), letting the existing jq check
('success' or 'skipped') decide pass/fail.
@sebastiw sebastiw changed the title Resurrect CI: pin Bazel 8.4.2, fix glob/sh_test defaults, fix CT/test regressions Fix github tests suites Jun 24, 2026

@lixen-wg2 lixen-wg2 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@sebastiw sebastiw merged commit 0608315 into develop Jun 24, 2026
10 checks passed
@sebastiw sebastiw deleted the fix/ci-resurrection branch June 24, 2026 11:56
sebastiw added a commit that referenced this pull request Jun 25, 2026
)

## Summary

Fix a Bazel analysis-time `conflicting actions` error in `eunit` when a
target's `test_deps` transitively depend back on the target's own app.

## Background

Commit e70ed5a (#5, "extract_app: namespace .beam/.app outputs by target
name")
intentionally writes `:erlang_app` and `:test_erlang_app` outputs to
distinct
paths (`<target>/ebin/foo.beam`) so both can live in the same package.
The
`ErlangAppInfo` provider was deliberately left unchanged, on the
assumption
that downstream rules consume the provider and would be unaffected.

`private/eunit.bzl::_impl` however builds its dependency list as:

```python
deps = list(ctx.attr.deps)
lib_info = ctx.attr.target[ErlangAppInfo]
deps.extend(lib_info.deps)
deps.append(ctx.attr.target)
```

If `ctx.attr.target` is `:test_erlang_app` for app `A`, and any entry in
`lib_info.deps` (i.e. the resolved `deps + test_deps` of
`:test_erlang_app`)
transitively pulls in `A`'s own `:erlang_app` via a `test_deps` cycle
(`A` test-depends on `B`, `B` depends on `A`), the resulting list
contains
two distinct `ErlangAppInfo`s sharing `app_name = "A"`.

`erl_libs_contents` keys its symlink destinations on `app_name`:

```python
dep_path = path_join(dir, lib_info.app_name)
...
dest = symlink(ctx, src, path_join(dep_path, "ebin", src.basename))
```

Before e70ed5a both source `.beam` files lived at the same execroot path
(`ebin/A.beam`) and Bazel coalesced the two symlink actions into one.
After the namespacing the source paths differ
(`erlang_app/ebin/A.beam` vs `test_erlang_app/ebin/A.beam`) but the
destination is still `<name>_deps/A/ebin/A.beam`, producing:

```
ERROR: file '.../eunit_deps/A/ebin/A.beam' is generated by these conflicting
actions:
  PrimaryInput: ... test_erlang_app/ebin/A.beam, ... erlang_app/ebin/A.beam
  PrimaryOutput: ... eunit_deps/A/ebin/A.beam
```

## Fix

De-duplicate the dependency list by `app_name` with the test target
winning,
by routing it through `flat_deps` (first-wins semantics) with
`ctx.attr.target` placed first:

```python
deps = flat_deps([ctx.attr.target] + list(ctx.attr.deps) + lib_info.deps)
```

This mirrors what the pre-e70ed5a action coalescing was effectively
doing:
the test variant's beams (a superset of the prod variant's) end up in
`ERL_LIBS`, and the prod variant pulled in transitively is suppressed.

`private/ct.bzl` is not affected because it never adds the test target
to
its dep list (test suites flow in via `compiled_suites`, not `deps`), so
no analogous change is needed there.

## Validation

- `cd test && bazelisk test //...` -> 13/13 pass.
- `cd examples/umbrella && bazelisk test //...` -> 9/9 pass.
- Reproduced and fixed the original failure against a downstream
consumer
  with two affected apps (cyclic `test_deps`):
- Before: `bazel build //app:eunit` fails analysis with the conflicting
    actions error shown above (for two distinct apps).
  - After: both apps build and their `eunit` targets pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants